Gradient matching methods for computational inference in mechanistic models for systems biology: a review and comparative analysis

Parameter inference in mathematical models of biological pathways, expressed as coupled ordinary differential equations (ODEs), is a challenging problem in contemporary systems biology. Conventional methods involve repeatedly solving the ODEs by numerical integration, which is computationally onerous and does not scale up to complex systems. Aimed at reducing the computational costs, new concepts based on gradient matching have recently been proposed in the computational statistics and machine learning literature. In a preliminary smoothing step, the time series data are interpolated; then, in a second step, the parameters of the ODEs are optimised so as to minimise some metric measuring the difference between the slopes of the tangents to the interpolants, and the time derivatives from the ODEs. In this way, the ODEs never have to be solved explicitly. This review provides a concise methodological overview of the current state-of-the-art methods for gradient matching in ODEs, followed by an empirical comparative evaluation based on a set of widely used and representative benchmark data

Controversy in mechanistic modelling with Gaussian processes

Parameter inference in mechanistic models based on non-affine differential equations is computationally onerous, and various faster alternatives based on gradient matching have been proposed. A particularly promising approach is based on nonparametric Bayesian modelling with Gaussian processes, which exploits the fact that a Gaussian process is closed under differentiation. However, two alternative paradigms have been proposed. The first paradigm, proposed at NIPS 2008 and AISTATS 2013, is based on a product of experts approach and a marginalization over the derivatives of the state variables. The second paradigm, proposed at ICML 2014, is based on a probabilistic generative model and a marginalization over the state variables. The claim has been made that this leads to better inference results. In the present article, we offer a new interpretation of the second paradigm, which highlights the underlying assumptions, approximations and limitations. In particular, we show that the second paradigm suffers from an intrinsic identifiability problem, which the first paradigm is not affected by

Estimating Parameters of Partial Differential Equations with Gradient Matching

Parameter inference in partial differential equations (PDEs) is a problem that many researchers are interested in. The conventional methods suffer from severe computational costs because these method require to solve the PDEs repeatedly by numerical integration. The concept of gradient matching have been proposed in order to reduce the computational complexity, which consists of two steps. First, the data are interpolated with certain smoothing methods. Then, the partial derivatives of the interpolants are calculated and the parameters are optimized to minimize the distance (measured by loss functions) between partial derivatives of interpolants and the PDE systems. In this article, we first studied the parameter inference accuracy of gradient matching based on two simple PDE models. Then the method of gradient matching was used to infer the parameters of PDE models describing cell movement and select the most appropriate model

Approximate parameter inference in systems biology using gradient matching: a comparative evaluation

Background: A challenging problem in current systems biology is that of parameter inference in biological pathways expressed as coupled ordinary differential equations (ODEs). Conventional methods that repeatedly numerically solve the ODEs have large associated computational costs. Aimed at reducing this cost, new concepts using gradient matching have been proposed, which bypass the need for numerical integration. This paper presents a recently established adaptive gradient matching approach, using Gaussian processes, combined with a parallel tempering scheme, and conducts a comparative evaluation with current state of the art methods used for parameter inference in ODEs. Among these contemporary methods is a technique based on reproducing kernel Hilbert spaces (RKHS). This has previously shown promising results for parameter estimation, but under lax experimental settings. We look at a range of scenarios to test the robustness of this method. We also change the approach of inferring the penalty parameter from AIC to cross validation to improve the stability of the method. Methodology: Methodology for the recently proposed adaptive gradient matching method using Gaussian processes, upon which we build our new method, is provided. Details of a competing method using reproducing kernel Hilbert spaces are also described here. Results: We conduct a comparative analysis for the methods described in this paper, using two benchmark ODE systems. The analyses are repeated under different experimental settings, to observe the sensitivity of the techniques. Conclusions: Our study reveals that for known noise variance, our proposed method based on Gaussian processes and parallel tempering achieves overall the best performance. When the noise variance is unknown, the RKHS method proves to be more robust

Model selection via marginal likelihood estimation by combining thermodynamic integration and gradient matching

Conducting statistical inference on systems described by ordinary differential equations (ODEs) is a challenging problem. Repeatedly numerically solving the system of equations incurs a high computational cost, making many methods based on explicitly solving the ODEs unsuitable in practice. Gradient matching methods were introduced in order to deal with the computational burden. These methods involve minimising the discrepancy between predicted gradients from the ODEs and those from a smooth interpolant. Work until now on gradient matching methods has focused on parameter inference. This paper considers the problem of model selection. We combine the method of thermodynamic integration to compute the log marginal likelihood with adaptive gradient matching using Gaussian processes, demonstrating that the method is robust and able to outperform BIC and WAIC

Statistical inference in mechanistic models: time warping for improved gradient matching

Inference in mechanistic models of non-linear differential equations is a challenging problem in current computational statistics. Due to the high computational costs of numerically solving the differential equations in every step of an iterative parameter adaptation scheme, approximate methods based on gradient matching have become popular. However, these methods critically depend on the smoothing scheme for function interpolation. The present article adapts an idea from manifold learning and demonstrates that a time warping approach aiming to homogenize intrinsic length scales can lead to a significant improvement in parameter estimation accuracy. We demonstrate the effectiveness of this scheme on noisy data from two dynamical systems with periodic limit cycle, a biopathway, and an application from soft-tissue mechanics. Our study also provides a comparative evaluation on a wide range of signal-to-noise ratios

Statistical inference for ordinary differential equations using gradient matching

A central objective of current systems biology research is explaining the interactions amongst components in biopathways. A standard approach is to view a biopathway as a network of biochemical reactions, which is modelled as a system of ordinary differential equations (ODEs). Conventional inference methods typically rely on searching the space of parameter values, and at each candidate, numerically solving the ODEs and comparing the output with that observed. After choosing an appropriate noise model, the form of the likelihood is defined, and a measure of similarity between the data signals and the signals described by the current set of ODE parameters can be calculated. This process is repeated, as part of either an iterative optimisation scheme or sampling procedure in order to estimate the parameters. However, the computational costs involved with repeatedly numerically solving the ODEs are usually high. Several authors have adopted approaches based on gradient matching, aiming to reduce this computational complexity. These approaches are based on the following two-step procedure. At the first step, interpolation is used to smooth the time series data, in order to avoid modelling noisy observations; in a second step, the kinetic parameters of the ODEs are either optimised or sampled, whilst minimising some metric measuring the difference between the slopes of the tangents to the interpolants, and the parameter-dependent time derivative from the ODEs. In this fashion, the ODEs never have to be numerically integrated, and the problem of inferring the typically unknown initial conditions of the system is removed, as it is not required for matching gradients. A downside to this two-step scheme is that the results of parameter inference are critically dependent on the quality of the initial interpolant. Alternatively, the ODEs can be allowed to regularise the interpolant and it has been demonstrated that it significantly improves the parameter inference accuracy and robustness with respect to noise. This thesis extends and develops methods of gradient matching for parameter inference and model selection in ODE systems in a systems biology context

Gaussian process emulation to accelerate parameter estimation in a mechanical model of the left ventricle: a critical step towards clinical end-user relevance

In recent years, we have witnessed substantial advances in the mathematical modelling of the biomechanical processes underlying the dynamics of the cardiac soft-tissue. Gao et al. (Gao et al. 2017 J. R. Soc. Interface14, 20170203 (doi:10.1098/rsif.2017.0203)) demonstrated that the parameters underlying the biomechanical model have diagnostic value for prognosticating the risk of myocardial infarction. However, the computational costs of parameter estimation are prohibitive when the goal lies in building real-time clinical decision support systems. This is due to the need to repeatedly solve the mathematical equations numerically using finite-element discretization during an iterative optimization routine. The present article presents a method for accelerating the inference of the constitutive parameters by using statistical emulation with Gaussian processes. We demonstrate how the computational costs can be reduced by about three orders of magnitude, with hardly any loss in accuracy, and we assess various alternative techniques in a comparative evaluation study based on simulated data obtained by solving the left ventricular model with the finite-element method, and real magnetic resonance images data for a human volunteer

Statistical Emulation of Cardiac Mechanics: an Important Step Towards a Clinical Decision Support System

No abstract available